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Abstract 

In this paper we compute the leading coefficients fi(u, w) of the 
Kazhdan-Lusztig polynomials P u , w for an affine Weyl group of type 
F>2- By using the a- function of a Coxeter group defined by Lusztig 
(see [LI, §2]), we compute most fj,(u,w) explicitly. With part of these 
values fi(u,w), we show that a conjecture of Lusztig on distinguished 
involutions is true for an affine Weyl group of type E>2 ■ We also show 
that the conjectural formula in [L3, (12)] needs a modification. 



The Kazhdan-Lusztig polynomials of a Coxeter group W play a central role 
in Kazhdan-Lusztig theory. From [KL1, (2.2.c)] one sees that the leading 
coefficients /j,(u, w) of some Kazhdan-Lusztig polynomials P UtW are very im- 
portant in understanding the Kazhdan-Lusztig polynomials. Moreover, the 
coefficients are of great importance in representation theory and Lie theory, 
and are related to some cohomology groups and some difficult irreducible 
characters (see [A,CPS,S]). 

In [L3] Lusztig computes the leading coefficients for some Kazhdan- 
Lusztig polynomials of an affine Weyl group of type B 2 . In [S] for an affine 
Weyl group of type A 5 , some non-trivial leading coefficients are worked out. 
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McLarnan and Warrington showed that fj,(u,w) can be greater than 1 for 
a symmetric group (see [MW]). In [XI], Xi showed that when u < w and 
a(u) < a(w), then /i(u,w) < 1 if W is a symmetric group or an affine Weyl 
group of type A n . In [SX, Theorem 3.3], Scott and Xi showed that the lead- 
ing coefficient /i(u, w) of some Kazhdan-Lusztig polynomial P UtW of an affine 
Weyl group of type A n is n + 2 if n > 4. In [G], Green showed that when 
W is a Coxeter group of type A n _i {n > 3), then n(u,w) < 1 if u is fully 
commutative. In [J], Jones showed that fi(u,w) < 1 when to is a Deodhar 
element of a finite Weyl group. 

In this paper we compute the coefficients fi(u, w) for an affine Weyl group 
W of type B 2 . There are four two-sided cells in W: c e = {w G W | a(w) = 
0} = {e}, d = {w e W \ a(w) = 1}, c 2 = {w e W \ a(w) = 2}, and c = 
{w E W | a(w) = 4}, where e is the neutral element of W (see [LI, §11.2]). 
The main results are the following: 

(1) For any u < w in W with (u, w) ^ (c x c ) U (c x c±) U (c x c 2 ), the 
value fi(u,w) is determined . The values of n(u,w) are displayed in Sections 
3, 5, 7 and 8. When (u, w) G c x Co, the value |U(«, «;) can be computed by 
using a formula in the proof for [SX, Theorem 3.1], see [W] for details. 

(2) Using the values of /i(u,w), we prove that Lusztig's conjecture on 
distinguished involutions proposed in 1987 is true for W (see Theorem 2.2) 
and that the W-graph of type B 2 is non-locally finite (see Theorem 8.2 and 
Remark 8.3). 

(3) The conjectural formula for b\ t y in [L3, (12)] needs a modification 
(see Proposition 5.1 and Remark 5.3.) 

1 Preliminaries 

In this section we recall some basic facts about /i(w, w) which will be needed 
later. 

1.1 Basic definitions and conventions 

Let R be a root system and W its Weyl group. Denote by A the weight lattice 
of R and A r = ZR the root lattice. The semi-direct product W = W x A r 
is an affine Weyl group and W = W x A is an extended affine Weyl group. 

We shall denote by S the set of simple reflections of W. There is an abelian 
subgroup Q of W such that uS = Sou for any uj G Vt and W — K W. The 
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length function / of W and the Bruhat order <on W can be extended to W 
by setting 1(ojw) = l(w) and ojw < oj'u if and only if oj — oj' and w < u, 
where u, oj' are in Q and w, u are in W. 

Let ii be the Hecke algebra of (W, S) over A = Z [g^,<?~^] (q an in- 
determinate) with parameter q. By definition, H is a free ^4-module and 
has a basis {T w } wgl y, its multiplication is defined by the relations (T s — 
q)(T s + 1) = if s G S and T W T U = T wu if l(wu) = l(w) + l(u). Let 

l(w) 

C w = q ~ z^ u<w P u , w T u ,w e W he its Kazhdan-Lusztig basis, where 
Pu,w £ Z[g] are the Kazhdan-Lusztig polynomials. (Here the C w is just the 
C' w in [KL1].) The degree of P U)W is less than or equal to ^(l(w) — l(u) — 1) if 
u < w and P W;W = 1. See [X3, 1.6] for more details. The subalgebra H of "H 
generated by all T s (s G 5) is the Hecke algebra of (W : S). For oj in and 
w in VF, we have C uw = T W C W . Thus we have Pu U , ww = P u , w and Pu' u ,ww = 
for any different oj, oj' in Q and t/; in W. It is known that the coefficients 
of these polynomials are all non-negative (see [KL2, Corollary 5.6]). 

Write P u>w = yu(«, w)g5( / ( w )~ / ( M )^ 1 )+ lower degree terms. The coefficient 
fi(u, w) is very interesting, which can be seen even from the recursive for- 
mula for Kazhdan-Lusztig polynomials [KL1, (2.2.c)]. We call li(u,w) the 
Kazhdan-Lusztig coefficient of P u , w - Write u -< w if u < w and fi(u, w) ^ 0. 
Define fl(u, w) = /i(u, w) if u < w and /2(tt, w) = fi(w, u) if w < u. 

The following properties for Kazhdan-Lusztig polynomials are known (see 
[KL1]): 

(a) P u w = P u -i w -i. In particular, fi(u,w) = /i(u 1 ,w 1 ). 

(b) Let u, w G W, s G S be such that u < w, su > u, sw < w. Then u -< w 
if and only if w = su. Moreover, fj,(u, w) = 1 in this case. 

(c) Let u, w G W, s G S be such that u < w, us > u, ws < w. Then u -< w 
if and only if w = us. Moreover, u(u, w) = 1 in this case. 

We refer to [KL1] for the definition of the preorders <l, <r, <lr and of 
the equivalence relations ~ LR on W. The corresponding equivalence 

classes are called left cells, right cells, two-sided cells of W, respectively. For 
w G W, set C{w) = {s G S | sw < w} and lZ{w) = {s G S \ ws < w}. Then 
we have (see [KL1, Prop. 2.4]) 

(d) lZ(w) C lZ{u) if u <l w. In particular, lZ{w) = lZ{u) if u ~l 

(e) £(«;) C £(m) if -u <r w. In particular, £(w) = C{u) if -u w. 

The a-function on V4 7 is defined in [LI, §2], which is a useful tool to study 
cells of W . The following result is due to Springer (see [XI, 1.3(a)]) 

(f) If fl(u, w) is non-zero, then (i) a{u) < a(w) implies w <l u and w <r u; 
(ii) a{u) = a{w) implies that u ~l w or u ~# w. 
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Write C W C U = ^2 zeW /i w C z , where h w>UjZ G A. Following Lusztig and 
Springer, we define 8 w>UiZ and 7™ iU)2 by the following formula: h w ^ z = 

Iw^zQ 3 ^ + 8w,u,z<r^ J ~+ lower degree terms. 

We shall need a result of Lusztig. Fix a subset S' C S consisting of 
two elements si,s 2 such that s\s 2 has order 4 and we denote by W the 
subgroup generated by s 1 , s 2 . Each coset W'w can be decomposed into four 
parts: one consists of the unique element u of minimal length, one consists of 
the unique element u' of maximal length, one consists of the three elements 
Siu, s 2 siu, s±s 2 siu, and one consists of the three elements s 2 u, Sis 2 u, s 2 sis 2 u. 
The last two subsets are called (left) strings. We shall regard them as 
sequences (as above) rather than subsets. The following result is due to 
Lusztig (see [LI, (10.4.2)]). 

(g) Consider two strings ui,u 2 ,us and wi,w 2 ,ws (with respect to S'). Set 
Ojj = fl(ui,Wj) if S' fl C(ui) = S' fl C(wj) and = otherwise. Then the 
integers satisfy the identities: an = a 33 ,ai 3 = a 3 i,a 22 = an + ai 3 , and 

We can also define (right) strings and then have similar identities. 



1.2 The lowest two-sided cell 

In this subsection we collect some facts about the lowest two-sided cell of the 
extended affine Weyl group W. 

For any u = UiUi, w = u 2 Wi, cui,cu 2 G Q, Ui,u>i G W, we say that 
u <l w (respectively u <r w, respectively u <lr w) if U\ <l W\ (respectively 
ujiUiu^ 1 <r ^Wicu^ 1 , respectively u\ <lr wi) (see [X3, §1.11]). The left 
(respectively right, respectively two-sided) cells of W are defined as those of 
W. We also define a(uw) = a(w) for u G £1, w G W. 

It is known that (see [Shi]) Co = {w G W \ a(w) = l(wo)} is a two-sided 
cell, which is the lowest one for the partial order <lr, where w is the longest 
element of W . We call c the lowest two-sided cell of W. 

In [SX, §2.1], the authors gave a description of c . Let R + (respectively 
R~, respectively A) be the set of positive (respectively negative, respectively 
simple) roots in the root system R of Wq. The set of dominant weights A + 
is the set {z G A | l(zwo) = l(z) + 1(wq)}. For each simple root a we denote 
by s a the corresponding simple reflection in Wo and x a the corresponding 
fundamental weight. For each w G Wo, we set 

d w = w 1 1 x a . 

w(a)eR~ 
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Then c = {d w zw d u 1 | w, u G VFo, z G A + }. 

(a) An element t/; is in c if and only if w = uw u' for some u,u' G W such 
that l(uw u') = l(u) + i(w ) + l{u'), see [Shi]. 

2 A conjecture of Lusztig 

Let 8(z) be the degree of P SjZ , where e is the neutral element of W. Then 
actually one has l(z) - a{z) - 25(z) > (see [L2]). Set V t = {d G W \ 1(d) - 
a(d) — 25(d) = %}. The elements in D are called distinguished involutions of 
W. In [L2], Lusztig showed that T> is finite for affine Weyl groups. 

Lusztig has a conjecture to describe T>\ (given in 1987, see [Sh2]). 

Conjecture 2.1 Assume that W is a Weyl group or an affine Weyl group. 
Let z G W. Then z is in T>i if and only if there exists some d G Dq such that 
z ~Li? d and p,(z, d) ^ 0. 

The "only if part of the conjecture is true, this is due to Shi; see [Sh2]. 
Moreover, forzeW and d G V , if jl(z, d) ^ 0, z ~l_r d and z ^ L z~ l , it is 
not difficult to prove that z G T>\. This part is due to Springer; see [Sh2]. For 
a Weyl group Wq, Lusztig showed that Conjecture 2.1 holds whenever the 
complex representation of Wq provided by the two-sided cell of Wq containing 
z does not contain the irreducible representation of Wo of degree 512 for type 
E 7 or of degree 4096 for type E s (see [XI, Sh2]). Also Xi proved that this 
conjecture is true for an affine Weyl group of type A n ; see [X2, Theorem 1.5]. 

In section 6 we will prove the following result. 
Theorem 2.2 The Conjecture 2.1 is true for an affine Weyl group of type 
B 2 . 

3 The second highest two-sided cell 

In this section, we assume that (W, S) is an arbitrary affine Weyl group. The 
second highest two-sided cell c\ of W is described in [L4]. 

Proposition 3.1 We have c\ = {w G W \ a(w) = 1} 
= {e^wEW\w has a unique reduced expression}. 

Then we get following corollary. 

Corollary 3.2 

(i) For any w G Ci, = |7£(w)| = 1. 

(ii) Let w 6 q. Assume that C(w) = {s'} and 1Z(w) = {s"}, then w ~^ s' 
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and w ~l s" . 

(iii) For any u, w in c±, u ~l iy is equivalent to 7?.(-u) = ^(ly) and w ~r «; is 
equivalent C(u) = C(w). 

Proof. We only need to prove (ii). By assumption we have w = w's" 
for some w' G W and l(w) = l{w') + 1. Then we get w <l s" '. By [L2, 
Corollary 1.9(b)] and a(w) = a(s") = 1, we get w ~ L Similarly, we get 
that w^ 1 ~l s', then w ~# s' follows. □ 

Now we give the values of fi(u, w) for u, w G c±. 

Proposition 3.3 Assume that (W, S) is an irreducible affine Weyl group of 
type different from A 2 . Then for any u,w G c±, u < w, we have 



Proof. Let u < w be in c\. Assume that fi(u,w) ^ 0. By 1.1 (f) we have 
u ~l w or u w . If u ~i w but u oo R w, then by Corollary 3.2, 
there exists some s' G S such that s'u > u and s'w < w. By 1.1(b), we get 
that w = s'u and fi(u, w) — 1. Similarly we can get that /(u>) — /(-u) = 1 and 
/i(w, iw) = 1 if u w but u ~h w. 

To complete the proof we need to prove that fi(u, w) = for u < w in Ci 
satisfying £(m) = C(w) and 7?.(w) = 1Z(w). 

First we assume that W is not of type A 2 m, so that the Coxeter graph 
contains no odd cycles. By Proposition 3.1, it can be checked that l(w) —l{u) 
is even for any u, w in c\ satisfying C(u) = C(w) and lZ{u) = 1Z(w). We are 
done in this case. 

When W is of type Aim for m > 2, the result can be proved by using star 
operation defined in [KL1]. Let S = {s , s±, . . . , s n } be the Coxeter genera- 
tors, where n = 2m. We know that they satisfy relations (siS i+ i) 3 = 1, for 
< i < n — 1, (s„s ) 3 = 1 and SiSj = SjSi for 1 < \j — i\ < n — 1. Using sym- 
metry and star operations we are reduced to the case u = (sis 2 s 3 • • • s n s ) Zl 
for some l\ G N and w = (S1S2S3 • • • s n So) k or w = (siSoS n s„_i • • • S3S2) fc SiSo 
for some k G N such that l± < k, where N is the set of nonnegative inters. 
We can assume that k — li = 2k + 1 for some k G N. If w = (sis 2 s 3 • • • s n s ) k , 
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then by [KL1, Theorem 5.2], we get that 

Hiu, w) = //((sis 2 s 3 • • • s n s ) h , (SiS 2 S 3 • • • s n s ) k ) 

— ^( s lS 2 Ss • • • S n SQ, (S1S2S3 • • • S n S ) k 

— M s 0) s o( s l s 2S3 ■ ■ ■ S n So) k ll ) 
= /i(siS , (S1S2S3 • • •S n S ) k ~ h ) 

= fi(u',w'), 
where u' = s m s m _ l ■ ■ ■ s 2 s 1 s (s n ■ ■ ■ s 2 s 1 s ) h and 

w' = s m s m+ i • • • s„s (s 1 s 2 • • • s n s ) k . We have l(w') — l(u') = 1 since n = 
2m. Because m > 2 and w has a unique reduced expression, we have u' ^ 
w'. Thus we obtain that fi(u,w) = fj,(u',w') = in this case. If w — 
(sis s n ■ ■ ■ s 3 s 2 ) k sis , we can prove that fj,(u, w) = similarly. 

The proof is completed. □ 

Proposition 3.4 Let (W, S) be an irreducible affine Weyl group of type A 2 . 
Then for any u, w G c±, u < w, we have 

(i) If C{u) ± C(w) or H(u) ^ H(w), then 

u(uw) = I 1 ' tf -'(«) = !. 
1^0, otherwise. 

(ii) If = and 7£(u) = ^(w), then 

r 1, if in -/(«) = 3, 

v ' I 0, otherwise. 

Proof. Part (i) is obvious by Corollary 3.2 and 1.1 (b)-(c). Let Sq, s±, s 2 be 
the simple reflections of W, so that (so s i) 3 — ( s i s 2) 3 = ( s o s 2) 3 - 

Assume that C(u) = C(w) and TZ(u) = lZ(w). Without loss of generality, 
we can assume that u = (sis 2 so) m z for some z G {e, si,«iS2} and m G N. 
Then w = (siS 2 So) n z or w = (sis s 2 ) n v for some v G {e, s±, SiSo} and n G N. 
We may require that n — m = 2k + 1 for some nonnegative integer k to ensure 
that l{w) — l{u) is odd and u < w. 
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If w = (sis 2 s ) n z, then by [KL1, Theorem 4.2], we have 
H(u,w) = /i((sis 2 so) m , (sis 2 s ) n ) 

= V(S1S 2 S , (siS2So)" _m+1 ) 
= fl(s2So,S 2 S (s 1 S2S ) n ~ m ) 

= n(s , so(s lS2 so) n ' m ) 

= yU(siS , (siS 2 So) n ~ m ) 



= /x(s (s 2 sis ) i , So(siS2So) n m *) 

= /i(siSo(s2SlS ) i , (siS2So) n ~ m "*) 

= MMlSo) m , S 2S0 (siS2S ) n - m - i - 1 ) 

= yU(s 1 So(s 2 SlSo) fe , (siS 2 So) fc+1 )- 

If n — m — 1, then = and to) = /j,(siSo, sis 2 so) = 1. If n — ra > 1, 
then > and SiSo(s 2 Sis Q ) k ^ (sis 2 s ) k+1 . So in this case we have n{u, w) = 
fi(s 1 s (s 2 s 1 s ) k , (sis 2 s ) fc+1 ) = 0. 

Now suppose that w = {siSoS 2 ) n v for some v G {e, Si,SiSo}- We can 
assume that z = e, so that v = Sis . If k > 0, then by [KL1, Theorem 4.2], 
we have 

Ai(u,w) = /i((s 1 s 2 so) m , (sis s 2 ) n sis ) 

= /i(s (sis 2 s ) m , s s 2 (siS s 2 ) n_1 SiSo) 
= Ks 2 s (s 1 s 2 s ) m , S 2 (siSoS 2 ) n_1 SiS ) 

= A((SlS 2 S ) m+1 , (s 1 S S 2 ) n ~ 1 S 1 S ). 

This reduces the problem to the case k — 0, and we can prove that iy) = 
in this case because u ^ w. 

Thus (ii) is true. The proof is completed. □ 



4 Semilinear equations related to ji 

In [L3] Lusztig introduced some semilinear equations, which are useful for 
calculating some jj,(u,w). In this section we first recall the equations, then 
give some discussions to the type B 2 . 

We need some notations. Let R + be the subset of R containing all positive 
roots. For any subset i C R + , we set a\ = J2 a ei Set p = \oir+. Then 
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p is a dominant weight and w(p) — p G A r for any w G W . For any A G A r 

we set 

Note that the summation index i runs through all subsets i of R + such that 
A can be written as the sum of all elements in i. If 7^ A cannot be written 
as a sum of distinct positive roots, we set $(A) = 0. 

Let A+ be the set of dominate weights in the root lattice A r , that is 
A+ = A r n A+. For any A G A+, we set 

W A = {w G W I w(X) = A} 

and 

where v\ is the number of reflections of W A . For A and A' in A+ we define 
a x ,x> = —Y J (-l) l{w) H\' + p-w(X + p)). 

Convention: For any element in A, we will use the same notation when it is 
regarded as an element in W. For two elements in A, the operation between 
them will be written additively if they are regarded as elements in A and will 
be written multiplicatively when they are regarded as elements in W. 

Lusztig showed that there is a 1-1 correspondence between A+ and the 
set of W — Wo double cosets in W (an element A of A+ corresponds to the 
unique double coset W0AW0 containing it: see [L5, §2]). For each A G A+, 
there is a unique element m\ of minimal length and a unique element M\ of 
maximal length in Wo A Wo- We have A < A' (i.e. A' — A G NR + ) if and only 
if M\ < My (Bruhat order) for A, A' G A+. 

Set v = q?, p u , w = v l W- l WP U:W (v 2 ) G Z^- 1 } for u < w G W and 
Pu,w = for all other u, w in W. For two elements A, A' in A+, set 

bx,y= £ (-vf m ^ Pz , mxl . 
zeWoXWo 

By [L3, §5], we know 

(a) a A ,A' is zero unless A < A' and are equal to 1 when A = A'; when A < A' 
it belongs to v~ lr L[v~ 1 }. 
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(b) b\ \i is zero unless A < A' and is equal to 1 when A = A'; when A < A' 
it belongs to i; -1 Z[i; -1 ]. Also we have /j,(m\,m\i) = Res„ = o(&A,A')j where 
Res„=o(/) G Z denotes the coefficient of v' 1 in / G A = 7*[v, v -1 ]. 

Let - : A — > A be the ring involution such that v = i> _1 . The following 
lemma of Lusztig gives a way to compute b\ t \i inductively (see [L3, Proposi- 
tion 7]). 

Lemma 4.1 For any A, A" G A+, we have 



In the rest of this paper, i? is a root system of type B 2 . Then (W, S) is 
an affine Weyl group of type B 2 and W is an extended affine Weyl group 
associated with R. 

Let a and f3 be the long and short simple root of R respectively. Then 
the set R + of positive roots in R consists of the following four elements: a, 
(3, a + (3, a + 2(3. The fundamental dominant weights are 



So the set A + of dominant weights consists of the elements mx+ny, m,n EN. 
Also we have 

= {mx + 2ny \ m, n G N} = {ia + j(3 \ i, j G N, i < j < 2i}. 

Let s and t be the simple reflections in S corresponding to a and (3 
respectively. The Weyl group W is generated by s and t. The set S contains 
a unique element out of Wq, denoted by r. Then W is generated by r, s,t. 
We have rt = tr, (rs) 4 = (st) 4 = e. 

Define X\ (respectively X 2 ) to be the subsets of A+ consisting of elements 
mx, m > 1) (respectively 2my, m > 1). Let Y 1 (respectively Y 2 ) be the 
subsets of consisting of elements x + 2my, m > 1) (respectively nx + 2my, 
n > 2 and m > 1). Set X = XiUAs, Y = Y 1 UY 2 . We have A+ = AUFU{0}. 

The following result is needed in calculating a\ t y and b\^\. 

Proposition 4.2 Let A be in X U Y. We have 



(i) If A G X 1 then W A = {e,t}. If A G A 2 then W A = {e, s}. If A G Y then 



x = a + (3 and y — - 



a + (3. 
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if A = mx G Xi, 
if A = 2my G X 2 , 
if A = x + 2my G Y\ , 
if A = nx + 2my G Y 2 . 



Proof. For the action of a Weyl group on the corresponding weight lattice, a 
well known result says that the stabilizer of a dominant weight is generated by 
the simple reflections stablizing the dominant weight. The simple reflections 
in W are s, t and we have s(x) = x — a = 2y — x, s(y) = y, t(x) = x, t(y) = 
y — (3 = x — y. Thus (i) follows from these facts. Also (i) can be proved by 
a direct calculation since Wo only contains 8 elements. 

(ii) is just a consequence of (i), since v x is the number of reflections in 
Wq. We know that n x = v~ Vx J2 weW \ v 2l<yW \ thus (iii) can be checked by (i) 
and (ii). 

Now we prove (iv). When A G A+, in W we have Xwo = woX" 1 and 
l(Xwo) = l(wo) + /(A). For A G A+, there is a unique w G Wo such that (1) 
A = wm x , (2) /(A) = l(w) + l(m x ), (3) C{m x ) = {r}. 

In W we have x = stsr and y = ursr, where u> G Q satisfies out = rui. 
Thus y 2 = tstrsr = tsrtsr. 

If A = mx G Xl, then in W we have A = (stsr) m = stsr(stsr) m ~ 1 . 
Therefore m x = r{stsr) m ^ 1 . If A = 2my G X 2 , we get that A = (tstrsr)" 1 = 
tstrsr (tstrsr) 171-1 . So m x = rsr (tstrsr)" 1 " 1 = rsr(tsr) 2m ~ 2 , since tr = rt. 

The longest element in Wo is = stst. When A = x + 2my G Y 1 , in W 
we have A = stsr (tstrsr)" 1 = stsrtstrsr (tstrsr)" 1 " 1 . Since tr = rt, we get 
A = w or srtsr (tsrtsr)" 1 " 1 = worsrftsr) 2 " 1 " 1 . Hence m x = rsrftsr) 2 " 1 " 1 . 

Now assume that A = nx + 2my G Y 2 . Set Ai = x + 2my, \ 2 = (n — l)x. 
Then in W we have A = AiA 2 = worsr(tsr) 2m ~ 1 (stsr)"" 1 . Thus m x = 
rsr(tsr) 2m ~ 1 (stsr) n ~ 1 . Noting that rsrs = srsr, tsts = stst and tr = rt, we 
see m x = (rst) 2m ~ 1 (srst) n ~ 2 srsrtsr. Thus (iv) holds. 

Since for A G A+, in W we have \w = WoX" 1 and l(\w ) = l(w ) + /(A), 
so M x = Xstst. Thus we have M x = stsm x stst if A G Xi, M x = tstm x stst if 
A G X 2 . Also M x = ststm x stst if A G Y since A = wom x in this case. Hence 
(v) holds. The proof is complete. □ 



v + v" 1 , if A G X, 
^^=1 1, if A G F. 

(iv) 



r(stsr) 



m—l 



mx ' rsr(tsr) 2rn -\ 



rsr(tsr) 2r "~ 2 
rsr(tsr) 

(rst) 2m ~ 1 (srst) n ~ 2 srsrtsr, 

l) (-l)l(mx)-l(M x ) 



-1, ifAGX, 
1, if A G Y. 
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A simple computation leads to the following identities: 
$(0) = 1, 

$(«) = $(/3) = -v~ 2 , 
$(a + /3) = $(a + 2(3) = v~ A - v~ 2 , 
$(2a + 2/3) = $(2a + 3(3) = v~ A - v~ 6 , 
$(2a + /3) = $(a + 3/3) = v~\ 
$(3a + 3(3) = $(2a + 4/3) = -t>" 6 , 

$(3a + 4/3) = and $(A) = for all for all other A in A r . 

Using the above formulas we can compute a\ t y, which are needed in 
determining &a,a' next section. 

Proposition 4.3 Let A and A' be in A+. Assume that < A < A' and 
X' = ia + j(3(i<j< 2i). 

(i) If A'-A ^ a, (3, a+(3, a+2(3, a+3/3, 2a+(3, 2a+2(3, 2a+3(3, 2a+4/3, 3a+3(3 
or 3« + 4/3, we have a\ t \i = 0. 

(ii) If A' — A = a or (3, we have a A ,A' = — v~ 2 . 

0, when j = i, 

(hi) If A' — A = a + (3, we have a,\ : \i = { ~v~ 2 , when j — 2% — 1, 



(iv) If A' — A = a + 2/3, we have a\ : \> — < _ 4 > 



t>~ 4 — v~ 2 , otherwise. 

v~ 2 , when j = i + 1, 

f ~ 4 — v~ 2 , when j > i + 1. 



(v) If A' — A = a + 3/3 or 2a + /3, we have a Ai y = f 4 

(vi) If A' — A = 2a + 2/3, we have a A \> = I _f 

' I v — v 

(vii) If A' - A = 2a + 3/3, we have 



when j > i, 
when j = i. 



v , when j = 2i — 1 or j — i + 1, 

" A - v ~ i w- 4 - v- 6 , otherwise. 

(viii) If A' — A = 2a + 4/3 or 3a + 3/3, we have oa,a' = — v _6 . 

(ix) If A' — A = 3a + 4/3, we have aA,A' = ^" 8 - 

Proof. By definition, p is the half of the sum of positive roots in R + . So 
we have p = |a + 2(3. Thus p — s(p) = a, p — t(p) = (3, p — ts(p) = a + 3(3, 
p — st(p) = 2a + (3, p — sts(p) = 3a + 3/3, p — tst(p) = 2a + 4/3 and 
p — stst(p) = 3a + 4/3, since s(a) = —a, s((3) = a + (3, t(a) = a + 2(3 
and t((3) = —(3. Since < A G A+, it is easy to see that sts(\),tst(X) and 
stst(X) = —A are all less than 0. For a weight v = aa + b(3 in A+, set 
h(v) = a + b. Then we have 

h(X' + p-w(X + p)) = h(X' - w(X) + p - w(p)) > 8 

if w = sts, tst or stst. As a consequence, we have $(A' + p — w(X + p)) — 
when w = sts, tst or stst. 
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Now we are ready to compute a Aj y. 

(i) If A' — A = n/3 for n > 2, we have A = ia+(j — n){3. Then i+n < j < 2% 
since A, A' G A+ and < A < A'. Clearly we have $(A' - A) = 0. Note that 

A' + p - s(X + p) = (2i - j + n + l)a + n/3, 

X' + p - t(X + p) = (2j -2i-n+ l)/3, 
X' + p - ts(X + p) = (2i - j + n + l)a + (2i + n + 3)/3 

and 

X' + p- st{X + p) = (j - n + 2)a + (2j -2i-n + l)/3. 

We have 2i- j + n + l > 3 and 2i- j + n+1 > n, so $(A' + p-s(A + p)) = 0. 
Since 2j - 2% - n + 1 > n + 1 > 3, we get $(A' + p - t(X + p)) = 0. From 
2i + n + 3 > 3n + 3 > 9, one sees that $(A' + p - ts(X + p)) = 0. Finally 
j -n + 2 > n + 2 > 4 (note that % > n) implies that $(A' + p-st(A + p)) = 0. 

Therefore ci\ t \' = if A' — A = n/3 for n > 2. Similarly we have oa,a' = 
if A' — A = na for n > 2. 

Also, by a similar computation we see oa,a' = if A' — A is one of the 
following elements: na + j3 (n > 3), a + n/3 (n > 4), na + 2(3 (n > 3), 
2a + n(3 (n > 5), na + 3/3 (n > 4), 3a + n/3 (n > 5), na + 4/3 (n > 4), 
ma + n(3 {m > 4, n > 5). Part (i) is proved. 

(ii) If A' - A = (3, then A = ia + (j - l)/3. Since A, A' G A+ and < A < A', 
we have 2<i + l<j<2i and i > 1. Thus 

A' + p - s(A + p) = (2i — j + 2)a + /3, 

A' + p-t(A + p) = (2j-20/3, 
A' + p - ts(A + p) = (2i -] + 2)a + (2i + 4)/3 

and 

X' + p- st(X + p) = (j + l)a + (2j - 2i)(3. 

So we get $(A' - A) = -t;" 2 , $(A' + p - s(X + p)) = $(2a + /3) = if 
j = 2% and $(A' + p - s(A + p)) = $((2i - j + 2)a + /3) = if j < 2i, and 
$(A' + p - i(A + p)) = $(A' + p - is(A + p)) = $(A' + p - st(A + p)) = 0. 

Thus we get a A ,A' = v+ v v -i (— f~ 2 — v~ 4 ) = — v~ 2 when j = 2i and a A ,A' = 
— v~ 2 when j < 2i. That is, oa,a' = — v ~ 2 if A' — A = /3. 

Similarly we get oa,a' = — v ~ 2 if A' — A = a. 

Part (ii) is proved. For parts (iii)-(ix), the arguments are similar. □ 
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In [LI, §11.2], Lusztig described the left cells and two-sided cells of (W, S). 
For any subset J of S = {r,s,t}, we denote by W J the set of all w G W 
such that lZ(w) = J. In the following we use the elements in J to denote J. 
Then (W, S) has 16 left cells: A rs = W rs , A rt = A rs t, A s = A Tt s, A r = A s r, 
A st = W s \ A' rt = A st r, A' s = A' rt s, A t = A% B rt = W rt - (A rt U A' rt ), B s = 
B rt s, B r = B s r, B t = B s t, C T = W r - {A r U B r ), C t = W* - (A t U B t ), 
C S = W S - (A, U A' s U B s ), D 9 = W® = {e}. 

Set c e = Dq, ci = C r U C s U C t , c 2 = B r U B s U B t U B rt , and c = 
A r UA s U A' s UAU A rs U A^. U ^rt U A' rt . 

From [LI, §11.2], we know that c e , Ci, Cq exhaust two-sided cells of 
W. We have c e = {w e W \ a(w) = 0} = {e}, c± = {w e W \ a(w) = 
1}, c 2 = {w G W | a(w) = 2}, and c = {ui G | a(iw) = 4}. 

When u, w in c\ with u < w, the value //(it, tu) is given in Proposition 3.3. 
In the following sections we compute /i(u, w) for other pairs (u, w) except 
those in (c x c ) U (c x ci) U (c x c 2 ). 



5 Computing it;) for (it, w) G C2 x C2 

Set C/ = {e, s,ts,rs} and V = {e, s,st,sr}, then c 2 = {-urt(srt) m t> | w G 
[/, v G V, m G N}. We will compute /i(u,w) for it, to G c 2 by means of the 
semilinear equations in Section 4. To do this we first compute the &a,a" for 
A, A" G A+. Our main results in this section are Theorems 5.7, 5.8 and 5.9. 

(a) Let m be a positive integer and Aj = ma + (m + i)/3, i — 0, 1, m — 1. 
Then we have 




i; 1 , if i = m — 2 = 0, 
— v~ 2 , if i = m — 2 > 1, 
0, if0<i<m-3. 



Proof. By 4 (b) we have &\ m _i,A m _i — 1- By Proposition 4.3 (ii— i) , we have 

(1) a AijAi+1 = -i;^ 2 for < i < m - 2 and a Xl ,\ J = if j - i > 2. 

Since A = mi is in X and Aj = (m—i)x+2iy is in Y for i = 1, 2, m—1, 
by Proposition 4.2 we see 

(2) (-ly^J-HM^) i s _i if i = o and is 1 if i > 1, 

(3) 7r Ao = i> + i;" 1 and 7r A - = 1 if i > 1. 

Now using Lemma 4.1 for Aj (0 < % < m — 2), A m _i, and 4 (a), we get 
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u 2 &A i+ i,;u-i if 1 < i < m 



2 and 



(4) &A l ,A m _i - v 2 b Xl+u \ m _ 1 - b Xi ,x m - 

(5) -(V + ^ 1 )&A„,A ra _ 1 - U" 2 6ai,A„,_i = -(« + O^Am-i ~ U^Ai.A,, 

Assume m = 2. Since & Al , Al — 1) by (5) we have — (v + v _1 )fr A()iAl 
-(u+tT 1 )^^-?; 2 . So6 AojAl -t> = 6a ,Ai-v _1 , which follows that b XoM = v' 1 
by 4.2 (b). 



v- 2 = 



Now assume that m > 3. By (4) and 4 (b) we get &A m _ 2 ,A„ 



— v 



Applying 4 (b), (4) when i 



m 



3 > 1 and (5) when m = 3 we get that 



^A m _ 3 ,A m _i = 0. Now using induction on z, (4) and (5) we see easily that 
^Ai,A m _i = 0if0<z<m — 3. This is the last part of (a). □ 

(b) Let m > 2 be a positive integer. Set Aj = ma + (m + i)j3 and ji = 
(m — l)a + (m — 1 + i)/3 for % — 0, 1, m — 1. Then we have 



v 
0. 



if % — m 

if z = m 

if z = m 

if z = m 



1 > 2 or z 

2>1, 
3 = 0, 
3 > 1, 



m 



0, 



ifz = m— l = lorif0<z<m — 4. 



Proof. By Proposition 4.3, we have 



a 7i.7i+l " 


= o-y j-1 a = — f 2 


for < z < m — 2 and a 7 


,7,=0if j- 








if z = m — 3 = 0, 








■u 4 — v 2 , 


if z = m — 2 > 1 or i = 


m — 3 > 1, 




■I 




if z = m — 4, 








if z < m — 5 or i = m - 


-2 = 0; 








if z = m — 1 or i = m - 


-2 = 0, 






v A — v 2 , 


if i = m-2> 1, 




a 7i.^m-l 


■I 




if z = m — 3, 










if z < m — 4. 




Since 70 


= ( 


m — l)x and 


7 m _i = 2(m - l)y are in 


X and 7j = 


+ 2iy is 


in Y for z = 1,2, 


m — 2, we see 





(4) (_l)Hm 7i )-KM 7i ) = ( -!> if j " or m- 1, ^ 

[ 1, itl<z<m — 2 



(5) 7T 7i 



v + v 1 , if z = or m — 1 , 
1, ifl<z'<m-2. 



By (a), we have 6 AiiAm _ 1 = if < i < m-3, ^a^i = 1, &A m _ 2 ,A m _ 1 = 
v _1 if m = 2 and &A m _ 2 ,A m -i = — "W" 2 if m > 3. By the proof of (a), we have 
(_l)i(mA i )-i(Af Ai ) = tta. = 1 if i > 1, (-l)'("»A )-i(^A ) = _i and 7rAo = 
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Let £ = v + v 1 and rj = v 4 — v 2 , r] = v 4 — v 2 . Now using (l)-(5), 
Lemma 4.1 for 7, (0 < z < m — 1), A m _i, and 4 (a-b) we get 

(6) If to = 2, then 

(7) If to > 3, then 

-^7m-l,Am-l + l~V~ 2 = -(6 7m -l,A ra -l + 1 - V 2 , 

hm^Xm-l + V ~ 2 & lm -l,\ m -l -V 2 T] + V = ^7m-2,A m -l +<> 2 £& 7m _i ) A m -l -V^fj+fj, 

V-3,A m -i -^~ 2 ^7 m -2,A m _ 1 -V 2 T}+V- 4 = b ym _ 3:Xm ^ -V 2 b lm _^ Xm _ x -V~ 2 f]+V 4 

for to > 4, 

5~ , \ , — v~ 2 b^ , \ , — -u~ 2 = 6^ , a i — v 2 6^ , a i — for 
to > 5, 

^.Am-l - ^~\ !+ l,A m -! = ^,A m _x - U 2 6 7 i+l,Am-l for 1 < * < 171 ~ 5 > 

-^70,A m _i - ^~\i,A m -i + 1 + V" 4 = -^70,A m _i - ^ 2 &7i,A m _! + I + V 4 for 
TO = 3, 

-^70,A m _i - ^" 2 ^ 7 i,A m - 1 - V" 2 = -^7o,A m _i - V 2 & 7l ,A m _! ~ « 2 for TO = 4, 
-^70,A m _i - ^ _2 &7i,A m -i = -^70,A m _i - ^ 2 ^7i,A m -i f O r m > 5- 

Using 4 (b) and (6), we see & 7m _ 1 ,A m _ 1 = and b 7m _ 2 , Am _ 1 = v" 1 if to = 2. 

Assuming to > 3. Using 4 (b) and (7), we get 6 7m _ 1 ,A m _i = v' 1 ; b lm _ 2t x m _ x = 
v~ 4 - v- 2 ; 6 7m _ 3iAm _ 1 is -v" 3 if to = 3, is v~ 4 if to > 4; and 6 7l , Am _ 1 = for 
< i < to - 4. 

Thus (b) is proved. □ 

(c) Let to > 3 be a positive integer. Set ^ = (to — 2)ct + (to — 2 + for 
« = 0, 1, to — 2 and A m _i = too; + (2to — l)j3. Then we have 

if i — to — 2, 
if i = to — 3 = 0, 
if i = m — 3 > 1, 
if < i < to - 4. 

Proof. The proof is similar to the proof of (b). □ 

(d) Let A m _i = ma + (2to — l)/3 and A = na + n'/3, where m,n,n' are 
positive integers satisfying n < n' < 2n and 1 < n < to — 3. Then we have 

&A,A m _i = 0. 

Proof. We prove it by descending induction on the partial order < in A+. 
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First we show that &A,A m _i = for A = (m — 3)a + 2(m — 3)(3. 
Note that m > 4. By (a)-(c), we have the following observation. 

(1) ^m-3,A m _l — ~ V ) ^m-2,A m -l — — V J ^7m-3,A m -l ~~ ^ ' ^7m-2,A m -l ~~ 

f- 4 - t;- 2 , ^7 m _ x ,A m _i = u -1 , &A m -2,A m -! = -^~ 2 , here ^, ^, A; are as in 
(c)-(a). Moreover, &A',A m _i 7^ f° r other A' in A+ with A < A' < \ m -i- 

By Proposition 4.3, we have 

(2) a XjUm _ 3 = -v- 2 , a x , Vm _ 2 = v- 4 - v' 2 , a A , 7m _ 3 = v~ 4 - v~ 6 , a A , 7m _ 2 = 
v~ 4 , flA )7m -i = -t>~ 6 , «A,A m _ 2 = ^~ 8 , aA,A m _! = 0. 

For A' = 7j, Aj, the values of (— iy( m A')~'( M A') an d ^ are determined in 
the proofs of (a) and (b). For A' = Ui, A, it is easy to determine the values of 
^_iy(m x ,)-i(M x ,^ an( j n ^ us j n g Proposition 4.2. Using (1-2), Lemma 4.1 

and 4 (a-b), we get that &A,A m _i = for A = (m — 3)a + 2(m — 3)/3 if m > 4. 

Let ^ A G A+ be such that A < (m - 3) a + 2(m - 3)/3. We show 
that &A,A m _i = 0. The induction hypothesis says that &A',A m _i = for those 
A < A' < (m - 3)a + (2m - 6)(3. 

If A = na + 2n/3 e X 2 , then n < m — 4, m > 5. By Proposition 4.3, we 
have 

(3) (l\,v m - 3 = a \,v m - 2 = a A,7 m -3 = a A,7 m - 2 = a A,7 m -i — a A,A m _ 2 = a A,A m _! = 

if m - n > 5; and a x ,v m _ 3 = v~ 4 , a A ,„ m _ 2 = -v~ 6 , a A)7m _ 3 = v~ 8 , a A , 7m _ 2 = 
a A, 7m -! = «A,A m _ 2 = aA,A m _! = if m - n = 4. 

Using (1), induction hypothesis, (3), Lemma 4.1 and Proposition 4.2, we 
can get that &A,A m _i = 0. 

Similarly, for A = na + n(3 (1 < n < m — 3) or A = na + (n + i)/3 (1 < 

1 < n — 1, 1 < n < m — 3) we can prove that &A,A m _i = 0. 

The proof is finished. □ 

A reformulation of (a)-(d) is the following result. 

Proposition 5.1 Let A" = ma + (2m — 1)(3 e Yi for some integer m > 2. 
For any / A G such that A < A", we have 
(i) If A = na + 2n/3 e X 2 for some n > 2, then 




if n = m — 1, 
if n = m — 2, 
otherwise. 
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(ii) If A = na + n/3 e X ± , then 



v 1 , if n = 1 or 2 and m = 2, 

v~ 5 , if n = 1 and m = 3, 

' ; y ^ r ;! . if n = 2andm = 3, 

0, otherwise. 



(iii) If A = na + (2n - 1) /? G Y u then 



1, if n = m, 

v~ 4 — v ~ 2 , if n = m — 1, 

' A - A " = S -v- 6 , if n = m- 2, 

0, otherwise. 



(iv) If A = x n y 2n =(n' + n)a + (2ri + n)/3 e Y 2 , then 

— v~ 2 , if n = 2 and n' = m — 2, 
v . \ <, t>~ 4 , if n = 2 and n' = m — 3, 
0, otherwise. 

Similarly to (a)-(d), we have the following results (e)-(h). 

(e) Let m be a positive integer and A, = ma + (m + i)f3 for i — 0, 1, ...,m. 

if i = m, 
~ 3 , if i = m — 1, 

if < i < m - 2. 

(f) Let m > 2 be a positive integer. Set 7^ = (m — l)a + (m — 1 + z)/? for 
i = 0, 1, ra — 1 and A m = ma + 2m/?. Then we have 



(g) Let m > 3 be a positive integer. Set z/j = (m — 2) a + (m — 2 + z)/? for 
i = 0, 1, m — 2 and \ m = ma + 2m/?. Then we have 




— 4 —2 

V * - V , 


if i = m — 


1, 


I) -4 + V 2 , 


if i = m — 


2 = 0, 


-v~ 3 - v~ 5 , 


if i = m — 


2> 1, 


0, 


if < i < 


m — 3. 



b, 



—v 6 , if i — m — 2, 
0, if0<i<m-3. 



(h) Let A = m'ct + m"/? and A m = ma + 2m/? e X 2 , where m, m',m" be 
positive integers such that m' < m" < 2m' and 1 < m' < m — 3. Then we 
have 6 A ,A m = 0. 
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We can reformulate (e)-(h) as follows: 

Proposition 5.2 Let A" = ma + 2m/3 G X 2 for some integer m > 2. For 
any 7^ A G A+ such that A < A", we have that 
(i) If A = na + 2n/3 G X 2 for n > 2, then 



1, 


if n — m, 




— 4 —2 
V 4 — V , 


if n = m — 


1 




if n = m — 


2 


0, 


otherwise. 





(ii) If A = na + n/3 e Xi, then 



^ if 4 + v 2 , ifm = 2,n = l 

A,A " — I 0, otherwise. 



(hi) If A = na + (2n - G Y 1 for 2 < n < m, then 

{i; -1 + v _3 , if n = m, 
— f -3 — v~ 5 , if n = m — 1, 
0, otherwise. 

(iv) If A = x n y 2n ' = (n' + n)a + (2n' + n)(3 eY 2 , then 6 a ,a" = 0. 

Remark 5.3 In [L3, (12)], Lusztig gave a conjectural formula on b\ : \> which 
says it is likely that, in the case where dv(A') > 1 for all s' G I and A G A+, 
we have 

b x ,x> = (-l)W-'wi ^ (-1)'H$( W (A' - p) - (A - p)). 

By Proposition 5.1 (i), we see that this conjectural formula is not true. By 
[L3, (12)], we get that &( m _i) a+ (2m-2)0,ma+(2m-i)0 = 0, which contradicts to 
Proposition 5.1 (i). 

Now we can compute fi(u,w) for u,w G c 2 . In [L3], Lusztig computed 
fi(rsr(tsr) n ,rsr(tsr) m ) for n is odd and m is even. 

Proposition 5.4 Let m,n£N. If m < n then we have 



fj,(rsr(tsr) m ,rsr(tsr) n ) = 



1, if n — m = 1, 
0, otherwise. 



Proof. We first assume that n = 2k — 1 and m = 2k' for some k > k' G N. 
Let A" = (jfc + l)a + (2k + 1)0 and A = (k' + l)a + 2(k' + l)/3. Then we 
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get myr = rsr(tsr) 2k 1 = rsr(tsr) n and m\ = rsr(tsr) 2k ' = rsr(tsr) m by 
Proposition 4.2 (iv). By Proposition 5.1 (i), we get that 

( v-\ i£V = k-l, 
bx,x„= I -v- 3 , if V = k-2, 
[ 0, otherwise. 

Therefore, by the fact that n{m\,m\ii) equals to the coefficient of i> _1 in 
b\,y, we get that (x(m.\,m\/r) = fi(rsr(tsr) m ,rsr(tsr) n ) = 1 if and only if 
n — m — 1; otherwise n(rsr(tsr) m , rsr{tsr) n ) = 0. 

When n is even and m is odd, the result can be proved similarly by 
Proposition 5.2 (iii). If m — n is even, then jj,(rsr(tsr) m ,rsr(tsr) n ) = is 
trivial. We complete the proof of the proposition. □ 

Let u and w be elements in c<i- By 1.1 (f), we know that if fj,(u,w) ^ 
then u ~l w or u ~r w. First we compute fj,(u, w) for those u, w such that 
u ~l w and u ~r u>. After that we deal with the case u ~l w but it to 
and the case u w but u ~r iu. 

Lemma 5.5 Let m,n£R If m < n then we have 



n(r(tsr) m ,rsr(tsr) n ) = 



1 , if n — m = 1 , 
0, otherwise. 



Proof. We have two left strings with respect to S f = {r, s} : 

Ul = r(tsr) m , u 2 = sr(tsr) m , u 3 = rsr{tsr) m 

and 

wi = r(tsr) n , u>2 = sr(tsr) n , W3 = rsr{tsr) n . 
By 1.1 (g), we get that 

/j,(r(tsr) m , rsr(tsr) n ) = fi(rsr(tsr) m , r{tsr) n ). 

Since t G C(r(tsr) n )\C(rsr(tsr) m ), then we get the result using 1.1 (b). □ 
Lemma 5.6 Let m, n e N. If m < n then we have 



fj,(r(tsr) m ts, rsr{tsr) n ts) 



1, if n — m — 1, 
0, otherwise. 



Proof. The proof is very similar to that of Lemma 5.5 □ 

For any two integers m, n, we get two left strings with respect to S' = 
{r, s} : 

Ui = r(tsr) m , u-i = sr(tsr) m , U3 = rsr{tsr) m 
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and 



Wi = r(tsr) n , u>2 = sr(tsr) n , w 3 = rsr{tsr) n . 



By 1.1 (g), we have 



fj,(r(tsr) m ,r(tsr) n ) = ji{rsr{tsr) m ,rsr(tsr) n ) 



and 



fi(sr(tsr) m ,sr(tsr) n ) = fi(r(tsr) m ,ritsr) n ) + n(r(tsr) m ,rsr(tsr) n ). 

Similarly, we get fi(sr(tsr) m ts, sr(tsr) n ts) 

= /i(r(tsr) m ts, r(tsr) n ts) + ^(r(tsr) m ts, rsr(tsr) n ts) 

= fi(rsr(tsr) m ts, rsr(tsr) n ts) + fi(r(tsr) m ts, rsr(tsr) n ts) 

= I2((rst) m+1 rs, (rst) n+1 rs) + ^(r(tsr) m ts, rsr(tsr) n ts). 

By 1.1 (a), we have 

fi((rst) m+1 rs, (rst) n+1 rs) = n{sr(tsr) m+1 , sr(tsr) n+1 ). 

We also get the following identities: 

fi(r(tsr) m t,r(tsr) n t) 

= fj,(rsr(tsr) m t,rsr(tsr) n t) = fi((rst) m+1 r, (rst) n+l r) 
= fi(r(tsr) m+1 , r{tsr) n+1 ) = ^(rsr(tsr) m+1 , rsr(tsr) n+1 ), 

/i(rsr(tsr) m tst,rsr(tsr) n tst) = ii(rsr(tsr) m t,rsr(tsr) n t) 



/i(r(str) 
fj,(r(tsr) 



m+1 



■ 1 ,r(str) n+1 ) 
r, (rst) n+1 r) 
-\r(tsr) n+1 ) 



(regarded as right strings with respect to S' = {s, t}), 

H((rts) m+1 t,(rts) n+1 t) = n(r(tsr) m tst,r(tsr) n tst 

= n(r(tsr) m t,r(tsr) n t), 



fjL(sr(tsr) m t, sr(tsr) n t) 



fi(st(rst) m r, st(rst) n r) 
fi(st(rst) m rsr, st(rst) n rsr) 
fx(sr(tsr) m+ \sr(tsr) n+1 ), 



fi{sr(tsr) m tst, sr{tsr) n tst) = /i(sr(tsr) m t, sr(tsr) n t) 
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and 



fi((tsr) m tst, (tsr) n tst) = /j((tsr) m t,(tsr) n t) 

= ii(t(str) m ,t(str) n ) 
= n{(rts) m t,(rts) n t). 

With these identities, Proposition 5.4, Lemma 5.5 and Lemma 5.6, we 
obtain the following theorem. 

Theorem 5.7 For elements u,w G c 2 such that u < w, u ~l w and u ~# w, 
we have 

(i) If s is not in C{u) U TZ(u), then 



1, if l(w) - l(u) = 3, 
0, otherwise. 



(ii) If s G C{u) \ K{u) or s G K(u) \ C(u), then 
fi(u,w) = 



2, if l(w) - l(u) = 3, 
0, otherwise. 



(iii) If s G C(u) mi(u), then 



3, if l(w) - l(u) = 3, 
0, otherwise. 



Proof. The only thing to notice is the fact: for any u,w G c 2 , u ~l w 
lZ(u) = lZ{w) and u ~r «; <^ = C(w). □ 

In the following we compute n(u,w) for those u,w G c 2 satisfying (1) 
u < w, (2) u ~l «; and w u>, or « ~# w and u ^ L w. 

Theorem 5.8 Let u,w G c 2 with u < w. Assume that u ~ L w and w ^ 
We have 

(i) If C{w) £ C(u), then 



fi(u,w) 

(ii) If £(w) C £{u), then 
A*(u,«;) 



1, if l(w) -l(u) = 1, 
0, otherwise. 



1, if Z(w) -l(u)= 5, 
0, otherwise. 
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Proof. Obviously, we have the fact: for any u,w G c 2 , u ~ L w lZ{u) = 
TZ(w) and u ~# w £( u ) — C(w). By the assumption on u, w, we get 
R(u) = R(w) but L(u) 7^ L(w). The proof of (i) is trivial by 1.1(b). 

Now we assume that L(w) C L(u), thus £(it) = L(w) = {r} or 

{£}. Without loss of generality, we can assume that L(w) = {r}. Then 

u G {rt(str) m , rt(str) m s, rt(str) m sr, rt{str) m st \ m G N} 

w G {r(str) n , r(str) n s, r(str) n sr, r(str) n st \ n G N} 

If u — rt(str) m = r(tsr) m t, then w = r(str) n = rsr(tsr) n ~ 1 t. By 1.1 
(g), we get /j,(r(tsr) m t,rsr(tsr) n ~ 1 t) = n(rsr(tsr) m t,r(tsr) n ~ 1 t) . Since t G 
L(r(tsr) n ~ 1 t)\L(rsr(tsr) m t), then by 1.1 (b) we get that 

1, if n — m = 2, 



H(rsr(tsr) m t,r(tsr) n -H) 
Thus we get 

I2(rt(str) m ,r(str) n ) = 
Similarly, we have 

fj,(rt(str) m s,r(str) n s) = 



0, otherwise. 



1, if n — m = 2, 
0, otherwise. 



and 



fj,(rt(str) m sr, r(str) n sr) 
H(rt(str) m st,r(str) n st) 



1, 


if n — m = 


2, 




otherwise; 




1, 


if n — m - 


= 2, 


o, 


otherwise 




1, 


if n — m = 


= 2, 


o, 


otherwise. 





By the assumption, we know that in this case n — m = 2 is equivalent to 
l(w) — l(u) = 5, thus (ii) holds. □ 

Similarly, we have the following theorem. 

Theorem 5.9 Let u,w G c 2 with u < w. Assume that u ~# w and u r^ L w. 
We have 

(i) HR(w) £ then 

/x(it, it?) = 

(ii) If R(w) C then 

/i(w,«;) = 



1, if f(iu) - Z(u) 
0, otherwise. 



I, if Z(w) - Z(u) = 5, 
0, otherwise. 
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6 The proof of Theorem 2.2 



Now we can give the proof of Theorem 2.2. The only thing we need to prove 
is: for some d G T>o, if z ~l z~ l ~l d and fl(z, d) ^ 0, then z &T>±. 

\i d E C\ and //(z, d) 7^ 0, then by the proof of Theorem 3.3 we know that 
z ~l d and z^ 1 ~ L d cannot hold at the same time. Also, if d G Co and 
p,(z, d) 7^ 0, we know that z ~l g? and z^ 1 ~ L d cannot hold at the same 
time, too (see the proof of Theorem 3.1 in [SX]). 

Now we assume that d G c 2 and z ~^ d ~l z -1 . Then we get L(z) = 
R(z) = L(d). By Theorem 5.7, those 2; satisfying z ~£ ~l d and //(d, z) 7^ 
for some d G T>$ D C2 are just rtstr, strstrs,rstrstrsr,tstrstrsr. By some 
computations, we can check that these four elements are all in T>\. The values 
of the a-function and the length function on these elements are obvious. We 
just need to get the degree S(w) of polynomials P ejW for these w by using 
the formula in [KL1, (2.2.c)]. For example, we have P e , r tstr — Pr,rtstr — 
Ptr,rtstr = Pr,rstr + QPtr,rstr — Q + 1- Thus we have l(rtstr) = 5, a(rtstr) = 2 
and S (rtstr) = 1. Therefore rtstr G T>\. With the same method, we can 
check that strstrs,rstrstrsr,tstrstrsr are all in T>\. Therefore, Conjecture 
2.1 holds for an affine Weyl group of type B 2 . 

Remark 6.1 In fact, we can show that the set T>i is finite for an affine 
Weyl group of type B 2 by Theorem 3.3, Theorem 5.7 and some results about 
fx(u, w) for u < w and (u, w) G c x c . For more details see [W]. 

7 Computing fi(u, w) for a(u) < a(w) 

In this section we will compute /i(u, w) clearly for those u < w such that 
a(u) < a(w). We will see that in this case fi(u,w) < 1. 

Theorem 7.1 For u < w such that a(u) = 1 or 2 and a(w) = 4, we have 



Proof. By 1.1 (f), if fj,{u,w) ^ we have w <l u and w <r u. Then by 
1.1 (d) and (e), if fi(u,w) ^ we have R(u) C R(w) and L(u) C L(w). If 
R(u) C R(w) or L{u) C L(w), the result is obvious by 1.1(b) and (c). In the 
following we assume that i?(w) = and L(u) = L(w). 

(1) Assume that a(u) = 1. If = L(t«) = {s}, we have L(sw) = 

{r,t} : L(su) = {r} or {t} or 0. Then there exists some s' E S such that 
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s' G L(sw)\L(su). By [KL1, (2.2.c)], we have 

Py,,w Psu,sw H~ qPu,sw ^ ^ A^(^5 sw^qz( ( ) ( ) ) P u z . 

u<z-<sw 
sz<z 

Also we have P SUjS „, = P s ' su , s „, and P„ iSW = P rtUiSW . If s'sw = sw, we have 
/i(u,w) = 1. If £(u>) — /(«) > 1 we get degP SUiS „, < — — 1) and 

deg(gP„ iSW ) < — /(it) — 1). By the non-negativity of the coefficients of 

Kazhdan-Lusztig polynomials, we get fx(u,w) = if l(w) — /(it) ^ 1. 

If L(it) = = {r}, we can assume that L(ru) ^ 0, since if u — r 

the proof is similar to the above. We have L(ru) = L(rw) = {s} and 
a(ru) = 1, a(rio) = 4. By [KL1, (2.2.c)], we have 

P —p A_ a p _ \ i(i(™)-i(«) + l)p 

- 1 u,ui 1 ru,rw ~ H 1 u,rw / PV ' "V^ - 1 «,z - 

rz<z 

If /(to) — /(it) > 1, by the above proof, we get degP rUtrw < ^{l{w) — l(u) — 1). 
Moreover, P UyTW = P S u,rw, L{su) = L(rw) = {s}. But l(rw) —l(su) ^ 1, then 
we get deg(qP Ujrw ) < \{l{w) — l(u) — 1) by the above proof. Thus fi(u, w) = 
in this case. 

If L(u) = L(w) = {t}, the proof is similar to the case L(u) = L(w) = {r}. 

(2) Assume that a(u) = 2. The proof is similar to (1). The only thing 
to note is the following facts: If L(u) = L(w) = {r, t}, then there exists 
some s' G {r, t} such that L(s'w) = {s,s"}, where s' ^ s" G {r, t}, and 
L(s'u) = {s"}. We have P UjS ' w = P s "su,s'w, where s"su > su > u. Then we 
consider the identity 

Pu,w Ps'u,s'w QPu,s'w ^ ^ A*(^) ^ W^jq 2 ^ ^ ) ( ) ) Pu,z- 

u<z^s f w 
s' z<z 

If L(u) = L(w) = {s}, we have L(su) = L(sw) = {r, t}. This case is 
trivial. 

If L(u) = L(w) = {r}, we have L(ru) = L(rw) = {s}. If l(w) — /(it) > 1, 
we get P u ,rw = Psu, rw and L(su) = {r, s}. We consider the identity 

p _p i „p _ ..( z rw )A(Krw)-l(z)+l) p 

1 u,w 1 ru,rw ~ u,rw / ' "^y 1 u,z- 

u<z^rw 
rz<z 

If u(su, rw) 7^ 0, we take z = su in the sum, then the item 
H,{su,rw)q^ l<yW ^ l(yU ^ lS) appearing in the sum is killed by qP u ,rw 
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If L(u) = L(w) = {t}, the proof is similar to the case L{u) = L(w) = {r}. 
We complete the proof . □ 

We define a subset E of C\ x C2 by 
E = {(st, srtst), (sr, srtsr), (r, rsrtsr), (£, tsrtst), (rst, rsrtst), (tsr, tsrtsr)}. 
We have the following theorem. 

Theorem 7.2 For u < w such that a(u) = 1 and a(w) = 2, we have 



H{u,w) 



1, if l(w) - l{u) = 1 or (it, w) G E, 
0, otherwise. 



Proof. By 1.1 (f), if //(i/,iu) ^ we get R{u) C P(w) and L(u) C L(w). If 
-R(w) £ P(u>) or L(u) C L(iy) the result is obvious by 1.1(b) and (c). In the 
following we assume that L(u) = L(w) = {s'} and R(u) = R(w) = {s"}. 

(1) If s' = s, s" = r. We have ix = (stsr) m or it = sr(stsr) m \w = (str) n sr 
for some 1 < m, n G N, < m! G N. If it = (stsr) m ,w = (str) n sr for some 
1 < m, n G N, we have L(su) = {t}, L(sw) = {r, t}. Consider the identity 

Pu,w Psu,sw ~\~ qPu,sw ^ ^ A*(^) sw^qz^ ( ) ( ) ) P uz . 

u<z^sw 
sz<z 

We get P SUiSW = P rsu , sw and P„ iSW = P rta , sw . If i(iu) - i(w) = 1 (i.e. m = 
n = 1) we get /t(it, to) = 1. Otherwise, we have degP SUiS „, < |(/(io) — /(it) — 
1), deg(gP n , sw ) < §(/(«;) - l(u) - 1), thus fi(u,w) = 0. 

If it = sr(stsr) m ' ,w = (str) n sr for some < mf G N, 1 < n G N, we have 
L(su) = {r},L(sw) = {r,t}. We get P su<sw = P tS u,sw and P u>sw = P rtu ,sw 
In this case there is no m',n G N satisfying tsu = sw, thus degP SUtSW < 
\{l{w) — /(it) — 1). Also we get P T tu,sw — 1 if fn' — 0, n — 1. Otherwise, 
deg(gP„ iSW ) < |(/(«;)— /(it) — 1). Then we get //(it, «;) = 1 if it = sr, w — srtsr; 
otherwise //(it, to) = 0. 

(1') The proofs for cases (i) s' = s,s" = t; (ii) s' = r,s" = s and (hi) 
s' = t, s" = s are similar to (1). 

(2) If s' = r, s" = t. We have it = r(stsr) m st, w = r(str) n st for some 
< m G N and 1 < n G N. Then L(ru) = L(rw) = {s}. Consider the 
identity 

P =P + a P - u(z r?;; N )fl5( / ( rw )- / ( z )+ 1 ) P 

- 1 J ru,rw ~ u,rw / FA ; uy / ' ( l 2 «,z - 

By (1'), we know that fi(ru,rw) = 1 if m = and n = 1; fi(ru,rw) = 
otherwise. Moreover, P M)ru) = P su ,rw with su > u and a(sit) = 1. By (!'), we 
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know that jj,{su,rw) — 1 if m — 0, n — 1; otherwise n(su,rw) = 0. Then we 
get that fj,(u, w) — 1 if u — rst, w = rstrst; otherwise fi(u, w) = 0. 

(2') The case s' = t, s" = r is similar to (2). 

(3) If s' = s" = r. We get u = r(stsr) m ,w = r(str) n sr for some < 
m G N, 1 < n G N. If m = (i.e u = r) we get fi(r, w) — 1 if w — rsrtsr; 
otherwise fj,(r,w) = 0. In the following we assume that m > 1, thus we get 
L{ru) = L(rw) = {s}. By (1), we know that n(ru,rw) — 1 if m — n — 1; 
otherwise ^(rw, ra) = 0. Moreover, we have P Uiru , = P su ,rw with su > u and 
a(sit) = 1. By (1'), we know that /i(su,rw) — 1 if m — 0, n — 1; otherwise 

ra) = 0. Thus we get the result. 

(3') The case s' — s" — t is similar to (3). 

(4) If s' = s" = s we shall prove that fi(u,w) = 1 if l(w) — l(u) = 1, 
otherwise, fj,(u,w) = 0. 

We have u G {(stsr) m s, (stsr) m sts, sr(stsr) m s, sr(stsr) m sts \ < m G 
N} and to = (str) n s for some 1 < n G N. If u = s we can compute that 
fi(s,w) = easily. In the following we assume that u ^ s. 

If u — (stsr) m s,w = (str) n s for some 1 < m,n G N, we have L(su) = 
{t}, L(sw) = {r, t}. Then we get fj,(su, sw) = 1 if l(w) — l(u) = 1, otherwise, 
/i(su, sw) = 0. Also, we have P UtSW = P rtu , S w ^ 1, thus deg(qP UjSW ) < \(l(w)- 
l{u) — 1). So we get fj,(u,w) = 1 if l(w) — l{u) = 1, otherwise, fj,(u,w) = 0. 

If u G {(stsr) m sts, sr(stsr) m s, sr(stsr) m sts \ < m G N}, the proof is 
similar. We complete the proof of the theorem. □ 



8 Computing /j,(u, w) for (it, w) G C2 x ci 

In this section we compute fj,(u,w) clearly for a(w) = 2 and a(ty) = 1. We'll 
see that fi(u,w) < 1 in this case. 

Proposition 8.1 For any 1 < n, m G N, we have 
/j,(rsr(tsr) n , (rsts) m r) = 



\ 1, if n = l,m > 2, 
1 0, otherwise. 



Proof. The proof is similar to the proof of Proposition 5.4. In the following 

we assume that n is odd, otherwise fj,(rsr(tsr) n , (rsts) m r) = 0. Let A = 
+ (n + 2)13 = xy n+1 and A" = (m + l)a + (m + l)/3 = x m+ \ then we 
have ui\ = rsr{tsr) n and rriyr = (rsts) m r, where m\ and my are the unique 
elements of minimal length in WqAWq and WoA'Wo respectively. By Section 
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4.1, we know that n(rsr(tsr) n , (rsts) m r) = Res v=0 b x ,\» . Due to Lemma 4.1, 
Proposition 4.2 and Proposition 4.3, we can get that 

{v^ + v^ 3 , ifn=l,m = 2, 
v~ l — v ~ 5 , if n = 1, m > 3, 
0, otherwise. 

Then the result is obvious. □ 
Following this proposition we can get our main result in this section. 

Theorem 8.2 For u < w satisfying a(u) = 2 and a(w) = 1, we have 

(i) If L(w) = R(w) = {s}, then 

, \ _ / 1) ifu — srts, 
^ ' ' \ 0, otherwise. 

(ii) If L(w) = {s}, R(w) = {s'}, where s' = r or t, then 

, \ _ / 1) ifu — srtss' or sr£, 
\ 0, otherwise. 

If L(w) = {s'}, = {s}, where s' = r or t, then 

, \ _ J 1, if u — s' srts or rts, 
0, otherwise. 

(hi) If L(to) = R(w) = {s'}, where s' = r or t, w ^ tsrst and rstsr, then 

, . _ J 1, if w G {s'srtss', rtss', s'srt, r£}, 
^ ' ' 0, otherwise. 

If L(w) = R(w) = {s'}, where s' = r or t, w = tsrst or rstsr, then 

H(u,w) = 



1, if u G {rtss', s'srt, r£}, 
0, otherwise. 



(iv) If L(w) = {s'},R{w) = {s"}, where s' ^ s" G {r,t}, w ^ rst(srst) 2 and 
tsr(stsr) 2 , then 



/i(u,w) = 



1, if m G {s'srtss", rtss", s'srt, rt}, 
0, otherwise. 
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If L(w) = {s'},R(w) = {s"}, where s' ^ s" G {r, t}, w = rst(srst) 2 or 
tsr(stsr) 2 , then 



Proof. For any u < w satisfying a(u) = 2 and a(w) = 1, if /j,(u,w) 7^ 0, by 
1.1 (d, e, f) we get L(w) C L(u) and R(w) C i2(u). 

We prove (iii) first. We assume that L(w) = R(w) = {r}, then w = 
(rsts) m r for some integer m > 1 and L(if), /?(«) = {r} or {r, t}. If m = 1, the 
claim is very easy to prove. In the following we assume that m > 2. Consider 
the two left strings w.r.t {r,s}: U\ = r(tsr)",M 2 = sr(tsr) n ,-u 3 = rsr(tsr) n 
and W\ = s(tsrs) m_1 tsr = (stsr) m ,W2 = rs(tsrs) m ~ l tsr = (rsts) m r,Ws = 
srs(tsrs) m ~ 1 tsr = s(rsts) m r = (srst) m sr. 

By 1.1 (g), we get /i(ui,W2) = (1(113,102), i.e. /i(r(tsr) n , (rsts) m r) = 
n(rsr(tsr) n , (rsts) m r). Similarly, we get 

fi(rt(str) n , (rsts) m r = ^(r(str) n+1 , (rsts) m r) (left strings w.r.t {r, s}). When 
L(w) = R(w) = {t}, we can also get the similar result. Using the facts 
jj,{u,w) = /ifw" 1 , -u; -1 ) and Proposition 8.1, we can get (iii). 

We now prove (ii). Assume that L(w) = {s} and R(w) = {r}, then 
w G {(srst) m sr, (stsr) m \ m > 1}. By 1.1 (g), we get 

fj,((str) n sr, (stsr) m ) = /i((str) n sr, (srst) m sr) = n(rsr(tsr) n , (rsts) m r) (left 
strings w.r.t {r, s}) and ii((str) n sr, (srst) m sr) = ^((str) n , (srst) m sr) 
= fi((str) n sr, (srst) m sr) (right strings w.r.t {r, s}). Then (ii) holds with the 
same reason of (iii). 

The proofs of (i) and (iv) are very similar to those of (ii) and (iii). □ 

Remark 8.3 (1) From Theorem 8.2, we see that the W-graph of type B 2 is 
nonlocally finite. More specifically, we have some element u in W such that 
there are infinitely many elements w in W satisfying fi(u, w) 7^ 0. This is 
one of the main interests of [L3] (the definition of W-graph can be found in 



(2) For those u < w satisfying a(u) = 4 and a(w) = 1 or 2, we can only 
get part of the leading coefficients. For example, we have /i(u, w) = when 
u and w are both of the minimal length in their double cosets WoiiWo an d 
Wo^Wcb respectively. As a consequence, we determine all ji{u, w) for those 
u < w satisfying l(w) — l{u) < 3. 




1, if u G {rtsrt, s'srtss" , rtss" , s'srt, rt}, 
0, otherwise. 



[KL1]). 
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